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ABSTRACT 

Unmanned  vehicles  perform  critical  mission  functions.  Today,  fielded  unmanned  vehicles  have  restricted  operations  as 
a  single  asset  controlled  by  a  single  operator.  In  the  future,  however,  it  is  envisioned  that  multiple  unmanned  air, 
ground,  surface  and  underwater  vehicles  will  be  deployed  in  an  integrated  unmanned  (and  “manned”)  team  fashion  in 
order  to  more  effectively  execute  complex  mission  scenarios.  To  successfully  facilitate  this  transition  from  single 
platforms  to  an  integrated  unmanned  system  concept,  it  is  essential  to  first  develop  the  required  base  technologies  for 
multi-vehicle  mission  requirements,  as  well  as  test  and  evaluate  such  technologies  in  tightly  controlled  field 
experiments.  Under  such  conditions,  advances  in  unmanned  technologies  and  associated  system  configurations  can  be 
empirically  evaluated  and  quantitatively  measured  against  relevant  performance  metrics. 

A  series  of  field  experiments  will  be  conducted  for  unmanned  force  protection  system  applications.  A  basic  teaming 
scenario  is:  Unmanned  aerial  vehicles  (UAVs)  detect  a  target  of  interest  on  the  ground;  the  UAVs  cue  unmanned 
ground  vehicles  (UGVs)  to  the  area;  the  UGVs  provide  on-ground  evaluation  and  assessment;  and  the  team  of  UAVs 
and  UGVs  execute  the  appropriate  level  of  response.  This  paper  details  the  scenarios  and  the  technology  enablers  for 
experimentation  using  unmanned  protection  systems. 
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1.  PROBLEM  STATEMENT 

Numerous  questions  emerge  when  expanding  unmanned  systems  from  a  single  remotely-operated  vehicle  with  a 
dedicated  user  to  a  system  of  systems  covering  multiple  dimensions  of  the  battlespace  with  a  heterogeneous  task  force 
of  unmanned  vehicles  of  various  levels  of  autonomy.  One  area  of  particular  importance  is  what  type  of  common 
operational  picture  is  required  to  effectively  perform  integrated  air/ground  mission  tasks.  Should  a  single  integrated 
operating  interface  be  deployed?  Or  would  a  mission  commander  “global  view”  with  individual  UAV  and  UGV 
operator  stations  be  more  effective? 

Additionally,  it  is  important  to  note  that  captured  imagery  streams  will  be  different,  particularly  with  respect  to 
orientation.  Fixed-wing  UAV  imagery  is  captured  in  a  vertical  orientation,  either  at  nadir  or  in  an  oblique  orientation. 
Rotary-wing  UAV  imagery  is  primarily  oblique.  Conversely,  UGV  imagery  is  captured  in  an  essentially  horizontal 
orientation.  As  such,  how  can  these  differing  orientations  best  be  fused  into  a  common  operating  picture,  one  that 
permits  and  promotes  rapid  user  understanding  and  enhanced  decision  making?  It  is  also  critical  that  a  common 
operating  picture  is  geospatial  in  nature,  one  that  accurately  and  dynamically  places  unmanned  assets  and  detected 
targets  in  an  easily  geo-referenced  framework. 

A  virtual  three-dimensional  visual  display  for  human/robot  interaction  with  UGVs  is  illustrated  in  Figure  1.  Numerous 
experimental  studies  have  demonstrated  that  the  designed  “augmented  virtuality”  display  significantly  increases  user 
performance.  Should  such  a  display  format  be  pursued  as  a  common  user  interface  for  multiple  unmanned  vehicle  types, 
and  if  so,  is  it  even  possible?  It  is  these  and  other  pressing  questions  that  will  be  carefully  and  methodically  addressed  in 
a  performance-based  and  quantifiable  manner  during  the  execution  of  the  technology  development  and  experiments. 
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Figure  1.  Unmanned  Ground  Vehiele  Virtual  Display 

Of  particular  interest  in  this  endeavor  is  how  to  better  integrate  the  unique  capabilities  of  small  UAVs  and  UGVs,  thus 
forming  a  collaborative  UAV-UGV  team  that  can  effectively  and  jointly  perform  mission-specific  operations.  A  first 
step  in  this  effort  is  the  development  of  a  generic  operational  script  that  can  be  applied  across  differing  mission 
scenarios.  Accordingly,  a  generic  field  experiment  script  has  been  developed  that  includes  the  following  major  task 
elements  for  collaborative  UAV-UGV  teaming: 

•  Acquire  from  simultaneous,  heterogeneous  US  Army  Class  II/III  affordable  UAVs,  high-resolution  geospatial 
imagery  (e.g.,  still  and  video). 

•  Detect  and  geolocate  possible  target  signatures  via  real-time  sensors  or  via  temporal  changes  within  imagery. 

•  Deploy  unmanned  vehicles  (e.g.,  UGVs  and  unmanned  aerial  rotorcraft)  to  detected  targets  as  appropriate. 

•  Perform  collaborative  unmanned  system  search  and  detection  behaviors  to  confirm,  identify  and  precisely 
locate  specific  targets. 

•  Perform  required  response  tasks  (e.g.,  monitoring,  manipulation,  neutralization)  with  UGV  and/or  UGV  assets 
as  appropriate. 

Based  on  this  generic  script,  three  applicable  scenarios  were  generated  dealing  with  Route  Protection,  Urban 
Reconnaissance  and  Monitoring,  and  Perimeter  Protection.  These  proposed  experiments  will  leverage  heavily  upon 
ongoing  unmanned  aerial  vehicle  (UAV)  and  unmanned  ground  vehicle  (UGV)  work  at  the  Idaho  National  Laboratory 
(INL)  and  SPAWAR  Systems  Center,  San  Diego  (SSC  San  Diego). 


2.  CAPABILITIES 


The  technology  under  development  has  applications  for  a  broad  range  of  small  UGVs  and  several  classes  of  UAVs 
engaged  in  a  variety  of  mission  profiles.  The  ability  to  exchange  meaningful  information  at  the  appropriate  level 
between  and  among  unmanned  systems  with  vastly  different  sensor  suites  and  perspectives  will  enable  effective 
teaming  that  can  be  brought  to  bear  on  several  Department  of  Defense  (DoD)  and  Department  of  Energy  (DOE) 
programs.  The  potential  to  not  only  exchange  information,  but  also  provide  an  adaptive  umbrella  of  control,  will  allow 
unmanned  vehicles  to  support  one  another’s  capabilities  and  limitations  across  a  wide  variety  of  tasks  and  applications. 
The  potential  ability  to  represent  the  combined  knowledge  and  task  progress  of  these  disparate  vehicles  in  a  single, 
seamless  Common  Operator  Control  Unit  (OCU)  is  of  particular  importance  and  would  enable  joint  unmanned  vehicle 
technology  across  many  different  protection-oriented  tasks. 

SPAWAR  Systems  Center,  San  Diego  (SSC  San  Diego)  and  its  predecessor  organizations  (NRaD,  NOSC,  NUC,  etc.) 
have  been  involved  in  various  aspects  of  robotics  since  the  early  1960's.  SSC  San  Diego  is  designated  by  the  Office  of 
the  Secretary  of  Defense  (OSD)  as  the  Center  of  Excellence  for  Small  Robots.  Our  mission  is  to  provide  network- 
integrated  robotic  solutions  for  Command,  Control,  Communications,  Computer,  Intelligence,  Surveillance  and 
Reconnaissance  (C4ISR)  applications.  SSC  San  Diego  has  over  15  active  projects  for  UGVs,  UAVs,  unmanned  surface 
vehicles,  unattended  ground  sensors,  and  unattended  munitions  with  the  focus  on  C4ISR. 

The  INL’s  unmanned  vehicle  program,  consisting  of  small  UGVs  and  both  fixed-wing  and  rotorcraft  UAVs  (see  figure 
2),  is  currently  transitioning  from  a  technology-centric  and  individual  platform  focus,  to  an  unmanned  vehicle  system 
and  application-centric  focus.  Of  particular  interest  in  this  transitional  effort  is  the  integration  of  unmanned  ground  and 
air  assets  into  a  single  unmanned  system  concept. 


Figure  2.  Idaho  National  Laboratory  Unmanned  Aerial  Vehieles  and  Unmanned  Ground  Vehieles 

Under  a  Memorandum  of  Agreement,  INL  is  teaming  with  SSC  San  Diego  in  the  coordinated  development,  evaluation, 
and  transfer  of  robotics  technology.^  This  arrangement  has  two  obvious  advantages:  1)  The  INL  Robotics  Group,  with 
similar  objectives  and  experience,  can  augment  the  available  manpower  resources,  allowing  more  technology  options  to 
be  evaluated;  and  2)  active  DOE  involvement  opens  up  another  major  conduit  for  exporting  results  into  relevant  user 
applications.  In  addition  to  accelerating  research  and  development  efforts  as  outlined  in  OSD  Joint  Robotics  Program 
Master  Plan  FY  2004^  this  synergistic  teaming  will  also  expedite  progress  towards  functional  objectives  set  down  in 
doe’s  Critical  Technology  Roadmap  for  Robots  and  Intelligent  Machines,^  thus  benefiting  a  variety  of  DOE  missions 
(i.e.,  decontamination  and  decommissioning  tasks,  environmental  monitoring,  security  applications,  homeland  defense, 
critical  infrastructure  protection,  and  emergency  response). 

The  joint  effort  between  INL  and  SSC  San  Diego  is  advancing  research  into  cooperative  behaviors  for  UAV  -  UGV 
teams.  Data  from  the  experiments  will  be  used  to  develop  new  technologies.  Concepts  of  Operations  (CONOPS),  and 
Tactics,  Techniques  and  Procedures  (TTPs)  to  effectively  integrate  unmanned  systems  with  the  warfighter. 


3.  EXPERIMENTATION 


Each  generated  scenario  represents  a  specific  experiment  and  will  showcase  unique  technology  sets  and  unmanned 
vehicle  capabilities.  Note  that  each  successive  experiment  requires  greater  air/ground  collaboration  and  teaming,  thus 
placing  increasing  emphasis  on  the  concept  of  an  integrated  unmanned  “system”  in  a  graduated  developmental  manner. 

3.1  Route  Protection/Countermine  (Scheduled  for  FY-05) 

The  Route  Protection/Countermine  scenario  will  demonstrate  ground  change  detection  from  a  UAV  combined  with 
collaborative  UGV  behaviors  to  protect  a  10-kilometer  route  for  convoy  transit.  The  UAV  will  acquire  still  imagery  for 
analysis  to  identify  possible  anomalies.  The  UAVs  will  cue  multiple  UGVs  to  detect,  assess,  mark  and  neutralize 
targets.  Scenario  sequence: 

a.  Deploy  UAV  between  points  A  &  B  on  route.  Collect  two  sets  of  images  (24  hours  apart)  over  area. 

b.  Perform  change  detection  to  identify  and  locate  anomalies  that  indicate  a  possible  minefield  and  an  Improvised 
Explosive  Device  (lED). 

c.  Cue  UGVs  to  converge  on  two  locations:  One  a  minefield  and  the  other  lED. 

d.  Perform  collaborative  UGV  search  behaviors  to  detect,  assess  and  mark  location  of  targets. 

e.  Explosive  Ordnance  Disposal  (EOD)  robots  are  dispatched  to  manipulate  and  neutralize  detected  targets. 

Demonstrated  technologies  include: 

•  Time-sensitive  delivery  of  geo-registered  spatial  data 

•  High-resolution  still  imagery 

•  Collaborative  search  and  detection  behaviors 

•  Countermine  detection,  marking  and  neutralization 

3.2  Urban  Reconnaissance  and  Monitoring  (Scheduled  for  FY-05) 

Urban  terrain  will  be  explored  collaboratively  by  multiple  unmanned  vehicles  to  build  a  common  operating  picture  that 
fuses  overhead  imagery  with  ground-based  mapping.  Selected  buildings  will  be  mapped  and  monitored  both  externally 
(UAVs)  and  internally  (UGVs)  for  human  presence  over  time.  Scenario  sequence: 

a.  Fuse  geo-referenced  overhead  imagery  of  urban  landscape. 

b.  Identify  access  routes  to  selected  buildings. 

c.  Deploy  UGVs  to  building  via  updated  access  paths. 

d.  UGVs  build  interior  maps  using  Simultaneous  Localization  and  Mapping  (SLAM)  technology. 

e.  Unmanned  vehicles  monitor  the  building  for  human  presence  or  approach. 

Demonstrated  technologies  include: 

•  Real-time  electro-optical/infrared  (EO/IR)  identification,  fusion  and  monitoring 

•  Common  OCU  with  scalable  representation 

•  Indoor  /  outdoor  synchronized  positioning 

•  Augmented  virtuality  interactive  interface 

•  Real-time  SLAM 

•  Human  Presence  Detection 

3.3  Perimeter  Protection  (Scheduled  for  FY-06) 

Access  to  a  strategic  location  must  be  denied  using  collaboration  between  unmanned  vehicles  to  assess  threat  beyond 
perimeter,  predict  approach  vector,  and  intercept.  Unmanned  vehicle  coverage  will  adapt  to  maintain  perimeter 
surveillance  and  expeditiously  intercept  threats  at  perimeter.  Scenario  sequence: 

a.  UAVs  maintain  mosaic  database  of  area  surrounding  a  region  of  interest. 

b.  Unmanned  vehicles  detect  possible  threat  via  change-detection  techniques  and  real-time  monitoring. 

c.  Path  of  incoming  threat  is  predicted  and  UGVs  are  deployed  to  intercept  at  perimeter. 

d.  Incoming  threat  is  intercepted  at  perimeter.  UAVs  provide  real-time  monitoring. 

e.  UGVs  engage  potential  threat  and  neutralize  if  necessary. 


Demonstrated  technologies  include: 

•  Geo-referenced  mosaic  imagery 

•  Stabilization  and  tracking 

•  Probability-based  vectored  response 

•  Adaptive  coverage  behavior 

•  Unmanned  vehicle  collaborative  patrol 

Proposed  FY-05  experiments  will  be  conducted  in  late  summer  (August  -  September  2005  timeframe).  Technology 
development  efforts  will  be  completed  prior  to  conducting  individual  experiments.  Technologies  and  data  will  feed 
directly  into  Joint  Robotic  Program’s  Family  of  Integrated  Rapid  Response  Equipment,  Collaborative  Engagement 
Experiment,  and  Countermine  Behavior  projects. 

4.  TECHNOLOGY  ENABLERS 

To  optimally  support  each  experiment,  a  series  of  base  technology  development/integration  tasks  must  be  completed. 
Note  many  of  the  tasks  represent  the  customization,  maturation,  and  integration  of  already  ongoing  work,  both  within 
the  Idaho  National  Laboratory,  SSC  San  Diego,  and  various  partnerships.  Specific  tasks  include: 

Mature  and  port  intelligent  control  capabilities  to  small  unmanned  ground  vehicles.  Enhance  indoor  SLAM 
capabilities.  Extend  dynamic  autonomy  for  collaborative  behavior  and  control  development.  Transition 
advanced  interface  components  into  a  common  OCU  by  fusing  a  variety  of  intelligence  and  interface 
capabilities  (e.g.  SSC  San  Diego’s  Multi-robot  Operator  Control  Unit  (MOCU)  (see  Figure  3)  and  INL’s 
Cognitive  Collaborative  Workspace)  into  a  single,  universal  controller  for  a  variety  of  military  platforms.  Port 
autonomy  onto  militarized  manportable  unmanned  ground  vehicle. 


Figure  3.  SSC  San  Diego  Multi-robot  Operator  Control  Unit  (MOCU) 

Mature  existing  Class  II/III  UAV  technologies  and  capabilities  for  relevant  mission-specific  applications. 
Exploit  and  demonstrate  relevant  experiment- specific  UAV  mission  capabilities.  Integrate  existing  high- 


resolution  camera  technologies  with  developed  geo-registration  capabilities  for  spatial  detection  of  anomalous 
events/signatures.  Mature  video  mosaic  mapping  and  image  stabilization  for  advanced  situation  awareness  and 
collaborative  behavior  fusion  during  both  day  and  night  operations.  Implement  appropriate  UAV-to-UAV  and 
UAV-to-UGV  ad-hoc  communication  protocols  and  develop  collaborative  behaviors  for  fault  tolerant  / 
extended  range  communications.  Implement  advanced  health  monitoring  capabilities  to  ensure  robust  UAV 
operations. 

Customize  current  INL  Change-Detection  System  software  to  UAV/UGV-specific  mission  applications. 
Develop  UAV-specific  ImagePlanner  capability,  allowing  pre-mission  image  capture  planning.  Develop  UAV- 
specific  geo-referenced  imagery  acquisition  capability.  Embed  geo-referenced  metadata  into  georegistration 
alignment  algorithm.  Embed  auto-generated  difference  algorithm  in  ImageAnalyzer  window.  Embed  generated 
geo-referenced  imagery  in  Geographic  Information  Systems  (GIS)  framework.  Develop  relevant,  experiment- 
specific  change  signature  libraries. 

Develop  integrated  unmanned  vehicle  ground  control  station  to  allow  mission  planning/operation  for 
sequential  and  concurrent  UAV  /UGV  /  manned  operations  including  image  processing  and  analysis. 

Develop  field  experiment  performance  metrics  and  measurement  techniques.  Test  and  evaluate  unmanned 
vehicle  collaborative  behaviors,  capabilities,  and  operations  in  field  environments  to  support  designed 
experiments. 

A  key  outcome  of  this  work  is  a  better  understanding  of  how  various  technology-  and  task-specific  variables  affect 
performance  in  a  complex,  unmanned  vehicle  mission  scenario. 

5.  INTEROPERABILITY 

Technology  enablers  for  unmanned  protection  systems  will  employ  spiral  development  to  harvest  and  integrate 
innovations  from  the  research  community  to  move  robotics  technology  towards  a  common  interface  and  control 
architecture  for  use  throughout  the  military. 

This  effort  is  closely  aligned  with  the  Joint  Robotics  Program  (JRP)  Technology  Transfer  (see  Figure  4)  managed  by 
SSC  San  Diego  -  a  centralized  mechanism  of  the  JRP  for  harvesting  the  best  features  of  prior/ongoing  research  efforts 
into  an  optimized,  standardized  system  that  can  be  easily  ported  to  robotic  platforms  used  service-wide.  To  facilitate 
integration  and  ensure  the  success  of  ultimate  transfer  to  ongoing  programs,  the  intent  is  to  adapt  and  standardize  on  a 
reconfigurable  software  framework  that  can  be  easily  ported  from  one  robotic  system  to  another. 
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Figure  4.  JRP  Technology  Transfer 


In  an  attempt  to  exploit  the  best  features  of  what  has  already  been  done,  this  effort  will  build  upon  the  INL’s  Advanced 
Robotic  Control  Architecture,  and  combine  it  with  the  University  of  Southern  California  (USC)  Player  Device  Server. 
INL’s  architecture  currently  hosts  Object  Detection  &  Object  Avoidance  (ODOA),  Target-Following  behaviors,  and 
SLAM  algorithms.  USC’s  Player  device  server  is  not  a  control  architecture  per  se,  but  a  robot  device  server  that  allows 
control  algorithms  to  access  robot  devices  in  a  standard  way,  promoting  standardization  of  a  platform  sensor 
framework.  Player  is  currently  being  integrated  and  tested  with  INL’s  control  architecture  to  allow  porting  to  any 
robotic  platform. 

In  addition,  the  project  is  further  closely  connected  to  the  goals  and  efforts  of  the  SSC  San  Diego  Man  Portable 
Robotics  Systems  (MPRS)  program,  which  has  demonstrated  interoperability  through  standardization  of  robot  platforms 
and  payloads  (common  hardware).  MPRS  has  supported  Joint  Architecture  for  Unmanned  Systems  (JAUS) 
demonstrations  with  hardware  and  software.  MPRS  has  also  developed  a  common  wearable  OCU  for  the  US  Army 
Night  Vision  and  Electronic  Sensors  Directorate  (NVESD)  that  controls  the  Urban  Robot  (URBOT),  iRobot  Packbot, 
and  JAUS -compatible  systems.  Under  the  UGV-UAV  effort,  this  OCU  will  be  extended  and  matured  to  include  new 
capabilities  including  indoor/outdoor  operations,  collaborative  search  and  detection,  and  augmented  virtuality.  The 
Common  OCU  (see  Figure  5)  is  specifically  designed  with  interchangeable  input/output  (I/O)  devices  to  allow  for 
interoperability  with  as  many  different  systems  as  possible.  By  combining  the  Common  OCU  with  the  existing  INL 
interface  components,  the  interoperability  of  both  the  individual  robot  behaviors  and  the  interface  will  be  ensured. 


Figure  5.  SSC  San  Diego  Common  Operator  Control  Unit  (OCU) 

The  technology  maturation  and  transition  efforts  under  this  program  will  be  married  closely  with  the  JRP  Robotic 
Systems  Pool,  which  provides  Commercial-Off-The-Shelf  (COTS)  robotic  systems  to  support  Concept  Experimentation 
Programs  (CEPs)  and  Limited  Objective  Experiments  (LOEs),  allowing  users  to  develop  requirements,  operational  need 
statements,  and  TTPs.  The  Man  Transportable  Robotic  Systems  Operational  Requirements  Document  calls  for  a 
teleoperated  system  that  will  be  enhanced  to  perform  EOD  tasks.  EOD  assets  from  the  pool  will  be  incorporated  into  the 
scenarios,  with  the  goal  to  develop  autonomous  behaviors  that  can  be  effectively  ported  and  transitioned  to  various 
military  COTS  systems  of  the  present  and  future. 

The  experiments  will  also  take  advantage  of  the  work  being  done  at  SSC  San  Diego  for  the  JRP  Collaborative 
Engagement  Experiment  (CEE)  to  develop  and  demonstrate  UGV-UAV  collaborative  technologies  in  a  Joint 
environment.  CEE  partners  are  SSC  San  Diego,  US  Army  Aviation  and  Missile  Research,  Development  and 
Engineering  Center,  and  Air  Force  Research  Laboratory  Robotics  Group.  SSC  San  Diego  is  utilizing  small  rotorcraft 
UAV  platforms  to  demonstrate  automated  launch,  refueling  and  precision  landing  mission  capabilities  (see  Figure  6) 
and  show  integration  with  a  common  unmanned  systems  control  system.  The  resulting  technologies  will  be  integrated 
into  the  Urban  Reconnaissance  and  Monitoring  and  Perimeter  Protection  phases  of  this  effort. 


Figure  6.  Automated  Launeh,  Refueling  and  Preeision  Landing  Mission  Capabilities  for  Rotoreraft  UAV 

6.  CONCLUSION 

The  Idaho  National  Laboratory,  SSC  San  Diego,  and  various  partnerships  are  teaming  for  spiral  development  on 
technology  enablers  for  unmanned  protection  systems.  The  teaming  effort  is  to  harvest  and  integrate  innovations  from 
the  research  community  to  move  robotics  technology  towards  a  common  C4ISR  interface  and  control  architecture  for 
use  throughout  the  military.  A  series  of  base  technology  development/integration  tasks  will  be  completed  and 
demonstrated  in  three  applicable  scenarios:  Route  Protection/Countermine,  Urban  Reconnaissance  and  Monitoring,  and 
Perimeter  Protection.  A  key  outcome  of  this  work  is  a  better  understanding  of  how  various  technology-  and  task- 
specific  variables  affect  performance  in  a  complex,  unmanned  vehicle  mission  scenario.  The  ability  to  not  only 
exchange  information,  but  provide  an  adaptive  umbrella  of  control,  will  allow  unmanned  vehicles  to  support  one 
another’s  capabilities  and  limitations  across  a  wide  variety  of  tasks  and  applications.  The  goal  is  to  expand  unmanned 
systems  from  a  single  remotely-operated  vehicle  with  a  dedicated  user  to  a  system  of  systems  covering  multiple 
dimensions  of  the  battlespace  with  a  heterogeneous  unmanned/manned  task  force. 
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